Harvestman: a framework for hierarchical feature learning and selection from whole genome sequencing data

Abstract Background Supervised learning from high-throughput sequencing data presents many challenges. For one, the curse of dimensionality often leads to overfitting as well issues with scalability. This can bring about inaccurate models or those that require extensive compute time and resources. Additionally, variant calls may not be optimal encoding for a given task, which also contributes poor predictive capabilities. To address these issues, we present Harvestman , method takes advantage hierarchical relationships among possible biological interpretations representations genomic variants perform automatic feature learning, selection, model building. Results We demonstrate scales thousands genomes comprising more than 84 million by processing phase 3 1000 Genomes Project, one largest publicly available collection whole genome sequences. Using breast cancer The Cancer Genome Atlas, show selects rich combination are adapted performs better binary representation SNPs alone. compare existing selection methods our is parsimonious —it smaller less redundant subsets while maintaining accuracy resulting classifier. Conclusion approach supervised building call data. By knowledge graph over solving an integer linear program automatically optimally finds right variants. Compared other methods, faster features parsimoniously.